Vision-based tracking may be used to track two-dimensional (2D) locations of fiducials in an image, or track three dimensional (3D) positions of fiducials using two or more cameras (outside-looking-in), or for wide-area tracking by placing fiducials in the environment (for example on ceiling) and mounting one or more cameras on the tracked object (vehicle, camera, person, etc.) and solving for the object's position and/or orientation based on the 2D bearings from the cameras to the fiducials and the known locations of the fiducials in the environment.
Such wide-area tracking capability might be used for mobile robots, augmented reality (AR), camera tracking (e.g. for television virtual sets, movie special effects), virtual reality walkthroughs, firefighter localization, AUTs, RPVs, UAVs, AGVs, etc. Most of these applications involve solving for the 6-DOF pose (position and orientation) of the tracked object, but some robotics or vehicular applications may only require the 2D position and the heading direction.
Augmented reality (AR) is a technology that merges virtual objects with the real world, typically by using a semi-transparent head-mounted display (HMD) to super-impose virtual objects on a user's view of the real world (called optical AR), or by using real and virtual cameras and merging their outputs on a display screen (called video AR). In video AR, a real video camera generates images of the real environment, and a virtual camera generates images of 3D graphics models to enhance a user 's perception of and interaction with the real world. Accurate registration between real and virtual scene elements needs to be achieved to make AR systems useful in real applications. To achieve this registration, it is necessary to accurately track the position and orientation of the optical see-through HMD or the real video camera in real time. Techniques for achieving accurate tracking can be generally divided into two areas, non-vision technology and vision-based technology. Vision-based AR systems detect fiducials or natural features using image processing or computer vision techniques to solve the registration problem.
One problem in AR is development of a tracking system that can provide high accuracy, very low latency, and a very wide tracking area. For many AR applications, including large-scale manufacturing, and outdoor applications, the area of operation is so large that a wearable computer must be used to drive the AR head-mounted display.
Motion tracking measures and reports an object's position and orientation as it changes in time. For example, when a user moves a computer mouse and the cursor mirrors that movement, the user is tracking motion in a very simple way. Motion tracking facilitates the interaction between people and computers. Historically, the lack of smooth and rapid tracking of the user's motion has blocked commercial acceptance of immersive 3D computer graphics. To be successful, the perspective of the virtual image seen by the computer must be an accurate representation of what is seen in the real world. As the user's head or camera moves, turns, or tilts, the computer-generated environment must change accordingly with no noticeable lag, jitter or distortion.